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Past  research  on  wearable  computers  for  maintenance  applications  has  focused  on  developing  displays  and 
presentation  formats.  This  study  emphasized  wearable  computer  control  technologies.  Alternative  control 
technologies  were  compared  with  standard  and  voice  controls.  Twelve  subjects  performed  a  synthetic 
maintenance  task  using  three  control  device  combinations  for  three  different  types  of  input.  Time  and  error 
data  were  collected.  The  results  show  that  for  pointer  movement,  standard  controls  took  significantly 
longer  than  voice.  For  discrete  input,  standard  controls  required  significantly  more  time  than  voice  and 
alternative  controls.  However,  there  were  no  significant  time  differences  among  controllers  for  text  entry 
fill-in.  Error  results  showed  no  significant  differences.  This  research  suggests  that  alternative  and  voice 
controls  provide  similar  performance  levels  and  both  are  superior  to  standard  controls.  In  environments 
with  changing  noise  spectra  and  noise  levels  such  as  a  flight  line,  the  alternative  control  suite  provides 
hands-free  control  that  complements  voice  without  sacrificing  performance. 


INTRODUCTION 

The  development  of  wearable  computer  systems  for 
maintenance  applications  promises  to  improve  technician 
performance  and  reduce  aircraft  downtime  by  readily 
providing  the  required  repair/modification  information 
wherever  the  technician  may  be.  However,  many  maintenance 
activities  require  the  use  of  both  of  the  technician’s  hands, 
therefore,  conventional  wearable  input  devices  are  inadequate 
for  computer  control.  For  example,  if  both  hands  are  required 
to  perform  a  wire  continuity  check,  it  is  difficult  to  enter  the 
obtained  value  via  a  wrist- worn  keyboard. 

While  voice  recognition  systems  offer  hands-free 
control,  they  can  be  problematic  in  environments  with 
changing  noise  spectra  and  noise  levels  (Chapman  & 

Simmons,  1995).  In  these  environments,  alternative  hands¬ 
free  controllers  are  required  that  are  resilient  to  the  specific 
constraints  associated  with  the  maintenance  technician’s  work 
environment. 

The  Air  Force  Research  Laboratory  Alternative 
Control  Technology  program  (AFRL/HECP)  is  developing 
several  hands-free  devices  for  computer  control.  The  devices 
utilize  signals  derived  from  the  brain,  muscles,  voice,  lips, 
head  position,  eye  position,  and  facial  gestures  (Calhoun  & 
Janson,  1991;  Jennings  &  Ruck,  1995;  Junker,  Berg, 


Schneider  &  McMillan,  1995;  McMillan,  Eggleston  & 
Anderson,  1997;  Nasman,  Calhoun  &  McMillan,  1997). 

These  computer  controllers  can  replace  conventional  systems 
or  they  can  be  used  as  a  supplement  to  standard  manual  input 
devices  or  voice  controls.  In  this  manner,  the  combined 
potential  of  conventional  and  evolving  alternative  controls  can 
be  exploited  to  provide  maintainers  with  a  variety  of  new 
channels  for  controlling  electronic  devices.  For  instance, 
keypad  control  may  be  more  efficient  for  filling  out  forms 
while  a  hands-free  controller  would  be  more  appropriate  for 
sequencing  through  procedures  during  manually  intensive 
wire  continuity  tests. 

To  date  the  primary  focus  of  research  and 
development  on  wearable  computers  by  the  Air  Force 
Research  Laboratory  Logistics  Readiness  Branch 
(AFRL/HESR)  has  been  on  display  development  and 
presentation  formats  (Friend  &  Grinstead,  1992;  Kancler  & 
Quill,  1997;  Masquelier,  1991;  Revels,  Quill,  Kancler  & 
Masquelier,  1998;  Webb,  1997).  However,  to  fully  exploit  the 
potential  advantages  of  the  wearable  computer  as  a 
maintenance  aiding  system,  research  and  development  must 
integrate  hands-free  control  technology. 

Under  normal  circumstances,  the  user  must  be  able  to 
provide  several  types  of  input  to  the  wearable  system.  In  the 
two  dimensional,  MS  Windows  operating  environment,  there 


are  three  primary  types  of  input:  (1)  pointer  movement  -  the 
positioning  of  the  cursor  on  the  screen  (commonly  performed 
by  moving  a  mouse),  (2)  discrete  input  -  the  selection  of  an 
object  of  focus  (e.g.,  the  left  mouse  button  click),  and  (3)  text 
entry  fill-in  (e.g.,  standard  keyboard  entry). 

The  current  study  compared  three  control  suites:  a 
standard  control  suite,  a  voice-only  control  suite,  and  an 
alternative  control  suite.  Performance  was  measured  for  three 
types  of  control  inputs  (pointer  movement,  discrete,  and  fill- 
in)  in  a  synthetic  maintenance  task  environment. 

METHOD 

Participants 

Twelve  male  maintenance  and  engineering  specialists 
from  the  Air  Force  Research  Laboratory  served  as  subjects. 
Each  subject  was  required  to  have  corrected  20/20  vision.  All 
subjects  were  under  the  age  of  forty. 

Apparatus 

A  monocular  head-mounted  Kopin  display  (HMD) 
was  used  to  display  the  presentation  software  in  all  conditions. 
The  display  was  black  on  white  monochrome  with  VGA 
resolution  (640  x  480  pixels)  and  subtended  approximately 
30°  of  visual  angle. 

Three  control  device  suites  were  used.  The  standard 
control  device  suite  integrated  a  thumbelina  miniature  track 
ball  and  a  wrist- worn  keyboard.  The  voice-only  control  suite 
used  the  Verbex  Voice  Systems  Rev  4.0  with  a  head  worn 
microphone.  Finally,  the  alternative  control  suite  incorporated 
a  head-mounted  Gyropoint  Pro  II  inertial  mouse  (for  head- 
based  cursor  movement),  the  CyberLink  electromyographic 
controller  (for  discrete  control  using  facial  gestures),  and  the 
Verbex  Voice  Systems  Rev  4.0  for  text  entry. 

Procedure 

The  task  was  a  hands-busy  maintenance  cannon  plug 
check.  For  this  task,  subjects  used  their  control  suites  to 
manipulate  between  screens  on  the  computer  display  while 
using  a  multimeter  to  measure  resistance  values  between  a 
given  set  of  pins  in  the  cannon  plug. 

Each  session  began  with  calibration  of  the  CyberLink 
and  speaker-dependent  Verbex  systems.  This  was  followed  by 
control  device  training  for  each  suite  of  controls,  which  served 
to  familiarize  subjects  with  both  the  control  suites  and  the 
HMD.  Finally,  each  subject  was  trained  on  the  cannon  plug 
task  using  the  control  devices  and  the  presentation  software. 
During  training,  a  single,  rectangular  cannon  plug  assembly 
was  used.  For  testing,  four  circular  cannon  plugs  were  used; 
each  plug  contained  either  12,  13,  55,  or  79  “female” 
(recessed)  pins. 


suites  for  a  total  of  48  trials.  Within  each  trial  the  subject 
performed  all  three  of  the  input  types  (pointer  movement, 
discrete,  and  fill-in).  A  typical  trial  was  as  follows: 

To  start,  the  subject  used  the  first  screen  to  determine  which 
cannon  plug  needed  to  be  tested  (the  appropriate  plug  was 
marked  by  a  small  gray  button;  see  Figure  1).  The  subject 
then  grasped  the  correct  plug.  Next,  the  subject  needed  to 
change  to  the  subsequent  screen  via  pointer  movement.  This 
entailed  moving  the  pointer  cursor  from  the  center  of  the 
display  to  within  the  border  of  the  gray  button.  No  discrete 
input  was  required.  For  the  standard  control  suite,  the 
thumbelina  trackball  was  used.  For  the  alternative  control 
suite,  the  head  tracker  was  used.  For  the  voice-only  condition, 
the  subject  had  to  say  the  number  of  the  plug  to  be  tested. 


it  (rmPointeiMove 


Select  Plug  Number:  55 


Figure  1 .  Example  of  a  Pointer  Movement  screen 

Next,  the  subject  had  to  insert  test  adapters  into  the 
appropriate  cannon  plug  pins  (see  Figure  2)  and  change  to  the 
last  screen  via  a  discrete  input.  For  the  standard  controls,  this 
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The  subjects  performed  sixteen  trials  (four  trials  for  Figure  2.  Example  of  Discrete  input  screen, 

each  of  the  four  plugs)  for  each  of  the  three  control  device 


entailed  clicking  the  left  button  on  the  thumbelina.  For  the 
alternative  control  suite,  discrete  input  was  performed  by  a 
facial  gesture  (such  as  a  short  jaw  clench  or  eyebrow  lift)  that 
was  relayed  to  the  system  using  the  CyberLink.  For  the  voice- 
only  condition,  the  subject  said,  “OK”. 

Finally,  the  multimeter  was  used  to  check  the 
continuity  of  the  circuit  between  the  two  pins  and  the  value  on 
the  meter  was  entered  on  the  fill-in  screen  (Figure  3)  using 
either  the  keyboard  (standard  control  suite)  or  voice  (voice- 
only  and  alternative  control  suites).  Four  different  resistance 
levels  were  used  for  the  meter  readings.  If  the  subject  did  not 
enter  the  correct  resistance  value  on  the  fill-in  screen,  an  error 
was  logged. 


Enter  Continuity  Values: 


jjgastartl  ,  ,1  cxfawn  _ I  *  ‘ m " 

Figure  3.  Example  of  a  Fill-in  screen. 


Type  [F(2,  22)=87.67,  p<.001].  The  interaction  of  Control 
Device  by  Input  Type  was  also  significant  [F(4, 44)= 12. 63, 
p<-001]. 

Figure  4  highlights  the  interaction  effect  using  mean 
time  in  seconds.  Using  the  Tukey  test  for  paired  comparisons, 
the  results  show  no  significant  differences  for  pointer 
movement  between  head  tracking  (the  alternative  control 
suite)  and  either  the  thumbelina  track  ball  (standard)  or  voice. 
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Figure  4.  Interaction  of  Control  Device  by  Input  Type. 


However,  the  standard  thumbelina  was  significantly  slower 
than  voice.  For  discrete  input,  it  took  significantly  longer  to 
make  a  selection  with  the  standard  thumbelina  than  it  did  for 
either  the  voice  or  the  CyberLink  alternative  controller.  For 
text  entry,  there  were  no  differences  in  text  entry  times  for  any 
of  the  control  devices  (voice  or  keyboard). 


Experimental  Design 

The  study  was  a  3  x  3  within-subjects  randomized 
block  design.  The  independent  variables  were  Control  Device 
(Standard  Control,  Voice  Control,  Alternative  Control)  and 
Input  Type  (Pointer  Movement,  Discrete,  Fill-in).  Subjects 
received  all  possible  combinations  of  control  devices  and 
input  types. 

Dependent  variables  were  completion  time  and 
percent  errors.  Completion  time  was  defined  as  the  time 
elapsed  from  activation  of  the  control  device  to  next  control 
device  activation.  Errors  were  defined  as  incorrect  fill-in  text 
entries.  Throughout  the  study,  the  experimenter  recorded 
observational  information.  Following  the  test,  subjects 
completed  a  post-test  questionnaire. 

RESULTS 

Data  were  analyzed  using  a  repeated  measures 
analysis  of  variance  (ANOVA)  procedure. 

Task  Time 

Analysis  of  task  time  revealed  significant  main 
effects  for  Control  Device  [F(2,  22)=32.58,  pc.OOl]  and  Input 


Percent  Error 

There  were  no  significant  differences  in  percent 
errors  among  the  three  control  devices,  however,  the  subjects 
did  show  large  intersubject  variability  in  the  number  of  errors 
for  the  alternative  control  condition. 

DISCUSSION 

Task  Time 

The  major  result  of  this  study  was  the  interaction  of 
Control  Device  by  Input  Type  for  time.  Among  controllers, 
performance  differences  for  the  different  input  types  were 
related  to  the  amount  of  hands-busy  activity  required  by  the 
task. 

Only  small  differences  were  found  for  the  pointer 
movement  task  -  the  small  trackball  used  in  the  standard 
control  suite  took  significantly  longer  than  the  voice  control 
but  was  not  significantly  different  from  the  head  pointing 
control.  During  this  portion  of  the  maintenance  task,  only  one 
hand  was  required  for  the  task  itself;  therefore,  the  other  hand 
was  free  to  use  the  standard  control. 


In  contrast,  hands-on  maintenance  was  most  intensive 
during  the  screen  requiring  discrete  input.  While  holding  the 
cannon  plug,  the  subject  typically  needed  to  find  the 
appropriate  pins  on  the  plug,  insert  the  test  adapters  using  the 
thumb  and  forefinger,  and  then  insert  the  multimeter  leads  into 
the  ends  of  the  test  adapters.  At  this  point  a  discrete  input  was 
required  to  advance  the  screen  presentation.  In  the  standard 
condition,  subjects  had  to  free  up  one  hand  and  move  it  to  the 
controller  to  activate  it.  Therefore,  the  standard  control  took 
significantly  longer  to  activate  than  either  the  voice  or  hands¬ 
free  alternative  controllers. 

For  text  entry  fill-in,  which  was  the  least  hands-busy 
of  the  conditions,  there  were  no  significant  differences  among 
the  three  types  of  controllers.  However,  a  difference  was 
expected  because  keyboard  input  (which  was  used  in  the 
standard  control  suite)  tends  to  be  slower  than  voice  input 
(which  was  used  by  both  the  alternative  and  voice  control 
suites)  (McMillan,  Eggleston,  &  Anderson,  1997).  This  lack 
of  difference  may  be  due  to  the  fact  that  text  entry  was  limited 
to  short  resistance  readings  taken  from  the  multimeter  with  an 
acknowledgement  (a  typical  entry  was  “7.9. . .  OK”)  while 
McMillan  et  al.  (1997)  refer  to  tasks  requiring  more  complex 
text  entries  in  conjunction  with  other  manual  or  visual  tasks. 

Percent  Error 

While  differences  in  percent  errors  were  not  found 
for  the  three  control  device  suites,  there  are  some  important 
issues  to  discuss.  For  three  subjects  in  the  alternative  control 
condition,  25%  of  the  entries  contained  an  entry  error. 
According  to  experimenter  observation  notes,  the  majority  of 
these  errors  were  due  to  inadvertent  activation  of  the 
CyberLink  discrete  control.  As  the  subject  was  performing 
the  maintenance  action  of  inserting  the  test  adapters  and  taking 
the  multimeter  reading,  occasionally  a  facial  gesture  (electro¬ 
myographic  signal)  would  accidentally  activate  the  CyberLink 
causing  the  screen  to  advance.  Because  the  screen  showing 
the  correct  pin  contacts  would  disappear,  the  subjects  had 
trouble  remembering  which  pins  to  measure.  This  increased 
the  likelihood  that  the  wrong  pins  were  tested  and  erroneous 
readings  were  entered. 

CONCLUSIONS  AND 
RECOMMENDATIONS 

The  results  of  this  research  suggest  that  hands-free 
input  devices  are  equal  to  or  better  than  standard  manual 
controls.  For  the  discrete  input  task  used  in  this  experiment 
during  a  manually  intensive  step,  both  voice  and  the 
electromyographic  (EMG)  controls  were  better  than  the 
standard  control.  For  the  pointer  movement  task,  the  voice 
control  was  better  than  the  standard  control.  However,  the 
lack  of  significant  differences  between  the  standard  and  head- 
based  control  devices  indicates  that  head-based  control  is  a 
definite  candidate  pointer  control  when  the  noise  environment 
precludes  the  use  of  voice  recognizers.  Head-based  pointer 
movement  may  be  even  more  advantageous  for  tasks  in  which 


the  format  is  more  complex  such  that  pointer  positioning 
cannot  be  readily  made  by  voice  commands. 

In  cases  where  there  were  no  significant  differences 
across  input  devices,  the  results  suggest  that  the  active  control 
suite  can  be  tailored  to  better  tolerate  some  environmental 
constraint  or,  in  the  absence  of  constraints,  to  whatever  device 
the  operator  prefers.  For  instance,  according  to  the  post-study 
questionnaire,  subjects  in  the  present  study  prefer  a  voice-only 
control  suite  when  possible.  However,  for  task  environments 
with  changing  noise  spectra  and  levels,  there  are  non-speech 
hands-free  control  alternatives  that  provide  comparable 
control. 

Future  Research 

Further  research  in  this  area  is  in  progress.  An  Air 
Force  flight  line  maintenance  field  evaluation  is  planned 
whereby  the  alternative  control  suite  can  be  used  as  a 
supplement  to  the  voice  and  standard  control  suites. 

Subjective  feedback  to  this  combination  of  supplementary 
computer  controls  will  be  collected. 

Furthermore,  to  reduce  unintentional  activation  of  the 
EMG  control  device,  further  study  is  underway  on  advanced 
feature  detection  algorithms  to  enable  a  purposeful  EMG  input 
to  be  more  reliably  discriminated  from  an  inadvertent  input. 
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